The present invention relates to computer program isolation methods. Specifically, the invention is a method and system for establishing and maintaining a restricted operating environment for a computer program to prevent the program from exploiting bugs and/or data of another computer program which shares the same hardware, while at the same time allowing some form of restricted, well-controlled communication between the programs.
Contemporary computers rely on a special set of instructions which define an operating system (O/S) in order to provide an interface for computer programs and computer components such as the computer's memory and central processing unit (CPU). Current operating systems have multi-tasking capability which allows computer programs to run simultaneously, each program not having to wait for termination of another in order to execute instructions. Multi-tasking O/S's allow programs to execute simultaneously by allowing programs to share resources with other programs. For example, an operating system running multiple programs executing at the same time allows the programs to share the computer's CPU time. Programs which run on the same system, even if not simultaneously with other programs, share space on the same permanent storage medium. Programs which are executing simultaneously are presently able to place binaries and data in the same physical memory at the same time, limited to a certain degree by the O/S restrictions and policy, to the extent that these are properly implemented. Memory segments are shared by programs being serviced by the O/S, in the same manner. O/S resources, such as threads, process tables and memory segments, are shared by programs executing simultaneously as well.
While allowing programs to share resources has many benefits, there are resulting security related ramifications. Some programs can have problems in execution due to mistakes or bugs in the program's instructions, or from conflicts with other running programs, or from ill-formatted or mischievous input handed to them. Further, some programs have been circulated which have intentionally embedded mistakes in them so that the program runs astray and becomes a so-called computer virus. Whether by mistake, or by a programmer's malicious intent, many major problems can, and often do occur, which can be traced back to these untrusted programs whose execution results are unpredictable and undesired. These problems include allowing the untrusted or rogue program: to capitalize CPU time, leaving other programs without CPU time; to read, forge, write, delete or otherwise corrupt files created by other programs; to read, forge, write, delete or otherwise corrupt executable files of other programs; and to read and write memory locations used by other programs to thus corrupt execution of those programs. As above, this may be a result of an intentionally malicious code, or a bug in an innocent code, or bad input handed to a code, or malicious input handed to a code, or a combination of these.
An example of such a scenario, where a set of trusted program has to run concurrently with a second set of untrusted programs, is a computer connected to the Internet. In this case, the computer may run an O/S, with several user applications, together comprising the trusted set of programs, concurrently with an Internet browser, possibly requiring also the execution of downloaded code, such as Java applets, or EXE/COM executables, the latter programs comprising the untrusted set. Sometimes the origin of such program cannot be verified, therefore it may be suspected of being malicious; the browser, when browsing an ill-formatted web-site or a malicious web-site may be subject to inputs that attempt to corrupt its behavior, e.g. too large input streams causing buffer overflow and possible undesired execution of code. It is desired that the execution of the untrusted programs has the least effect on the trusted programs, and this effect should be controlled and confined to a restricted form through, for example, preset file or memory locations, specific interrupts, etc.
Many security features and products are being built by software manufacturers and by O/S programmers to prevent such breaches from taking place, and to ensure the correct level of isolation between programs. Among these are generic architectonic solutions such as rings-of-protection in which different trust levels are assigned to memory portions and tasks, paging which includes mapping of logical memory into physical portions or pages, allowing different tasks to have different mapping, with the pages having different trust levels, and segmentation which involves mapping logical memory into logical portions or segments, each segment having its own trust level wherein each task may reference a different set of segments. Since the sharing capabilities using traditional operating systems are extensive, so are the security features. However, the more complex the security mechanism is, the easier it is for a rogue program to bypass the security and to corrupt other programs or the operating system itself, sometimes using these very features that allow sharing and communication between programs to do so.
Further, regarding rogue or virus programs, for virtually every software security mechanism, a programmer has found a way to subvert, or hack around, the security system, allowing a rogue program to cause harm to other programs in the shared environment. This includes every operating system and even the Java language, which was designed to create a standard interface, or sandbox, for Internet downloadable programs or applets.
The vulnerability of computer programs lies in the architecture of the computer operating system itself. A typical prior art operating system scheme is depicted in FIG. 1. The traditional, multi-tasking O/S environment includes an O/S kernel 100 loaded in the computer random access memory (RAM) at start-up of the computer. The O/S kernel 100 is a minimal set of instructions which loads and off-loads resources and resource vectors into RAM as called upon by individual programs executing on the computer, generally indicated at 102. Sometimes, when two or more executing programs require the same resource, such as printer output, O/S kernel 100 leaves the resource loaded in RAM until all programs have finished with that resource. Other resources, such as disk read and write, are left in RAM while the operating system is running because such resources are more often used than others.
The inherent problem with the prior art architecture depicted in FIG. 1 is that resources, such as RAM, or disk, are shared by programs simultaneously, giving a rogue program a pipeline to access and corrupt other programs, or the O/S itself through the shared resource. Furthermore, as the applications that are to be used in the prior art are of general nature, many features are made enabled to them by the O/S, thus in many cases bypassing the O/S security mechanism. Such is the case when a device driver or daemon is run by the O/S in kernel mode, which enables it unrestricted access to all the resources. Corruption can thus occur system wide.